Method, system and non-transitory computer-readable recording medium for supporting object control

ABSTRACT

According to one aspect of the present disclosure, there is provided a method of assisting an object control, comprising the steps of: determining a first coordinate with reference to, as a trigger coordinate, a coordinate at a time point when a trigger event relating to movement of a control means is generated, among motion coordinates of the control means; determining a second coordinate with reference to at least one of a distance between the trigger coordinate and the motion coordinates, a straight line section specified by the trigger coordinate and the motion coordinates, a distance between the first coordinate and the motion coordinates, and a straight line section specified by the first coordinate and the motion coordinates; and determining a motion vector determined based on the first coordinate and the second coordinate as an instruction vector for determining a control position in a control object region.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to Korean Patent Application No.10-2020-0102814 filed on Aug. 14, 2020, Korean Patent Application No.10-2020-0121843 filed on Sep. 21, 2020, and Korean Patent ApplicationNo. 10-2020-0158512 filed on Nov. 24, 2020.

TECHNICAL FIELD

The present disclosure relates to a method, a system, and anon-transitory computer-readable recording medium for assisting objectcontrol.

BACKGROUND

Inevitable contact may occur in the course of touching a display,pressing a button, and turning a switch on and off. A user may touchwith his/her hand a place to which respiratory droplets sprayed from aperson infected with a virus (e.g., COVID-19) adhere. In this case, whenthe user touches his/her own mucosa or conjunctiva with his/her hand,the user may be infected with the virus.

In particular, they may come into contact with each another viaintermediations such as touch displays, buttons, switches, and the likelocated in a public place where an unspecified number of people isgathered. This increases the risk of infection. Thus, there is a needfor a method capable of controlling the intermediations in a non-contactmanner.

There exist various non-contact control manners using a proximitysensor, an infrared (IR) touch, a hand gesture, and the like. Unlike acontact control manner capable of specifying an accurate contactposition, the non-contact control manner may fail to accuratelyrecognize a region intended by a user. Thus, there is a need for anon-contact control manner to predict a position intended by the user.

As an example, a manner of selecting a coordinate closest to a controlobject region within a predetermined distance among coordinates may beused. However, when control means such as a fingertip of the user doesnot approach the control object region in a vertical direction to thecontrol object region, an error between a coordinate intended by theuser and a coordinate predicted by a system may occur.

As another example, a manner of connecting two coordinates of bodyportions of the user (e.g., a coordinate of the wrist and a coordinateof the fingertip) with each other to specify a control position may beused. However, this makes it difficult for the user to intuitivelyrecognize the control position, and thus a visual guide such as a cursorneeds to be provided.

As yet another example, a manner of connecting eye(s) and a fingertip ofthe user to specify a control position may be used. However, a guidanceon how to select a position at which the eyes and the fingertip areconnected to each other needs to be provided additionally to the user.In particular, there is a possibility that the closer the distancebetween the user and the control object region, the more often only thefingertip moves. This may result in a larger error.

As yet another example, a manner of performing control when a controlmeans comes close to a control object region within a predetermineddistance, may be used. However, in a system in which a trigger isdetected when the control means comes close to the control object regionwithin a predetermined distance (e.g., 5 cm), the user is unlikely toaccurately recognize a specific position spaced apart from the controlobject region by the predetermined distance. This makes it difficult forthe user to perform a delicate operation. Further, since the trigger maybe detected while the user is moving toward the control position, alocation that is completely different from the intended position of theuser may be selected. In particular, there is a problem in that when thecontrol means is brought closer to the control object region, theaccuracy of prediction grows higher, but the risk of contact also growshigher.

Meanwhile, the control operation of the user may be generally classifiedinto (i) motion of moving a fingertip toward a control position, and(ii) motion of touching a control object region with the fingertip so asto perform control.

In this case, the motion (i) of moving the fingertip toward the controlposition may vary depending on a position of the user, a position of theuser's hand before the control, or the like. For example, when the useris on the side of the control object region or when the user needs tolift up his/her hand to perform the control, the motion (i) of movingthe fingertip toward the control position and the motion (ii) oftouching the control object region with the fingertip are different indirection from each other so that the motions may be regarded asdistinct operations. That is, when the directions of the motion (i) ofmoving the fingertip toward the control position and the motion (ii) oftouching the control object region with the fingertip are different fromeach other, it becomes difficult to predict the final control positionthrough the motion (i) of moving the fingertip toward the controlposition. In this case, the control position may be predicted based onthe position of the fingertip at the time the motion (i) ends.

In addition, when the user performs control while being in front of thecontrol object region, the directions of the motion (i) of moving thefingertip toward the control position and the motion (ii) of touchingthe control object region with the fingertip are similar to each other,and these motions may be performed in a consecutive order. That is, whenthe directions of the motion (i) of moving the fingertip toward thecontrol position and the motion (ii) of touching the control objectregion with the fingertip are similar to each other, the final controlposition may be predicted based on the motion (i) of moving thefingertip toward the control position.

In short, the control operation is classified into the motion (i) ofmoving the fingertip toward the control position and the motion (ii) oftouching the control object region with the fingertip. In case that themotion (i) and the motion (ii) are performed in a consecutive order, thecontrol position is predicted based on the motions. In case that themotion (i) and the motion (ii) are distinguished from each other, thecontrol position is predicted based on the position of the fingertipafter the motion (i). Therefore, it is possible to more accuratelypredict the control position intended by the user.

On the other hand, by determining a control intention and a controlposition with reference to a time point when the user stops the controlmeans, the user can control with a recognition of his/her own controltime point. In particular, when the motion (i) and the motion (ii) areperformed in a consecutive order, it becomes possible to accuratelypredict the control position even at a position farther away from thecontrol object region.

Based on the above findings, the inventors present a novel and improvedtechnique which is capable of accurately specifying a control positionthat accords with the intention of a user in a control object region byspecifying a motion vector based on a first coordinate determined withreference to, as a trigger coordinate, a coordinate at a time point whena trigger event relating to movement of a control means is generatedamong the motion coordinates of the control means, and a secondcoordinate determined with reference to a distance and straight linesection specified based on the first coordinate or the triggercoordinate, and using the specified motion vector.

SUMMARY

One object of the present disclosure is to solve all the above-describedproblems.

Another object of the present disclosure is to accurately predict acontrol position that accords with the intention of a user in a controlobject region.

Yet another object of the present disclosure is to minimize occurrenceof error by verifying validity of a motion vector for specifying acontrol position.

Still another object of the present disclosure is to specify a controlposition intended by a user by dynamically determining a controlposition calculation manner having a relatively high accuracy among aplurality of control position calculation manners.

Representative configurations of the present disclosure to achieve theabove objects are described below.

According to one aspect of the present disclosure, there is provided amethod of assisting an object control, comprising the steps of:determining a first coordinate with reference to, as a triggercoordinate, a coordinate at a time point when a trigger event relatingto movement of a control means is generated, among motion coordinates ofthe control means; determining a second coordinate with reference to atleast one of a distance between the trigger coordinate and the motioncoordinates, a straight line section specified by the trigger coordinateand the motion coordinates, a distance between the first coordinate andthe motion coordinates, and a straight line section specified by thefirst coordinate and the motion coordinates; and determining a motionvector determined based on the first coordinate and the secondcoordinate as an instruction vector for determining a control positionin a control object region.

Further, according to another aspect of the present disclosure, there isprovided a system for assisting an object control, comprising: acoordinate management unit configured to determine a first coordinatewith reference to, as a trigger coordinate, a coordinate at a time pointwhen a trigger event relating to movement of a control means isgenerated, among motion coordinates of the control means, and configuredto determine a second coordinate with reference to at least one of adistance between the trigger coordinate and the motion coordinates, astraight line section specified by the trigger coordinate and the motioncoordinates, a distance between the first coordinate and the motioncoordinates, and a straight line section specified by the firstcoordinate and the motion coordinates; and an instruction vectormanagement unit configured to determine a motion vector determined basedon the first coordinate and the second coordinate as an instructionvector for determining a control position in a control object region.

In addition, there are further provided other methods and systems toimplement the present disclosure, as well as non-transitorycomputer-readable recording media having stored thereon computerprograms for executing the methods.

According to the present disclosure, it is possible to accuratelypredict a control position that accords with the intention of a user ina control object region.

Further, according to the present disclosure, it is possible to minimizeoccurrence of error by verifying validity of a motion vector forspecifying the control position.

Furthermore, according to the present disclosure, it is possible tospecify a control position intended by a user by dynamically determininga control position calculation manner having a relatively high accuracyamong a plurality of control position calculation manners.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustratively shows an internal configuration of an objectcontrol assistance system according to one embodiment of the presentdisclosure.

FIG. 2 illustratively shows a process of assisting object control to auser, according to one embodiment of the present disclosure.

FIG. 3 illustratively shows a process of assisting object control to auser, according to one embodiment of the present disclosure.

FIG. 4 illustratively shows a process of assisting object control to auser, according to one embodiment of the present disclosure.

FIG. 5 illustratively shows a process of assisting object control to auser, according to one embodiment of the present disclosure.

FIG. 6 illustratively shows a process of assisting object control to auser, according to one embodiment of the present disclosure.

FIG. 7 illustratively shows a process of assisting object control to auser, according to one embodiment of the present disclosure.

FIG. 8 illustratively shows a process of assisting object control to auser, according to one embodiment of the present disclosure.

FIG. 9 illustratively shows a process of assisting object control to auser, according to one embodiment of the present disclosure.

FIG. 10 illustratively shows a process of assisting object control to auser, according to one embodiment of the present disclosure.

FIG. 11 illustratively shows a process of assisting object control to auser, according to one embodiment of the present disclosure.

DETAILED DESCRIPTION

In the following detailed description of the present disclosure,references are made to the accompanying drawings that show, by way ofillustration, specific embodiments in which the present disclosure maybe practiced. These embodiments are described in sufficient detail toenable those skilled in the art to practice the present disclosure. Itis to be understood that the various embodiments of the presentdisclosure, although different from each other, are not necessarilymutually exclusive. For example, specific shapes, structures andcharacteristics described herein may be implemented as modified from oneembodiment to another without departing from the spirit and scope of thepresent disclosure. Furthermore, it shall be understood that thepositions or arrangements of individual elements within each of theembodiments may also be modified without departing from the spirit andscope of the present disclosure. Therefore, the following detaileddescription is not to be taken in a limiting sense, and the scope of thepresent disclosure is to be taken as encompassing the scope of theappended claims and all equivalents thereof. In the drawings, likereference numerals refer to the same or similar elements throughout theseveral views.

Hereinafter, various preferred embodiments of the present disclosurewill be described in detail with reference to the accompanying drawingsto enable those skilled in the art to easily implement the presentdisclosure.

Configuration of Object Control Assistance System

An internal configuration of an object control assistance system 100crucial for implementing the present disclosure and functions ofrespective components thereof will be described.

FIG. 1 illustratively shows an internal configuration of the objectcontrol assistance system 100 according to one embodiment of the presentdisclosure.

Referring to FIG. 1, the object control assistance system 100 accordingto one embodiment of the present disclosure may include a coordinatemanagement unit 110, an instruction vector management unit 120, acommunication unit 130, and a control unit 140. Further, according toone embodiment of the present disclosure, at least some of thecoordinate management unit 110, the instruction vector management unit120, the communication unit 130, and the control unit 140 may be programmodules to communicate with an external system (not shown). Such programmodules may be included in the object control assistance system 100 inthe form of operating systems, application program modules, and otherprogram modules, while they may be physically stored in a variety ofcommonly known storage devices. Further, the program modules may also bestored in a remote storage device that may communicate with the objectcontrol assistance system 100. Meanwhile, such program modules mayinclude, but not limited to, routines, subroutines, programs, objects,components, data structures, and the like for performing specific tasksor executing specific abstract data types as will be described belowaccording to the present disclosure.

Although the object control assistance system 100 is described as above,such a description is an example. As will be understood by those skilledin the art, at least some of the components or functions of the objectcontrol assistance system 100 may be implemented inside or included in adevice (to be described below) as needed. In addition, in some cases,all functions and all components of the object control assistance system100 may be executed entirely inside the device or may be includedentirely in the device.

The device according to one embodiment of the present disclosure is adigital device having a memory means and a microprocessor for computingcapabilities, and may include a wearable device such as smart glasses, asmart watch, a smart band, a smart ring, a smart necklace, a smartearset, a smart earphone, a smart earring, or the like, or a somewhattraditional device such as a smart phone, a smart pad, a desktopcomputer, a server, a notebook computer, a workstation, a personaldigital assistant (PDA), a web pad, a mobile phone, a remote controller,or the like. The device may be changed in various forms at such a levelthat can achieve the objects of the present disclosure as well as theforegoing examples. Further, the device according to one embodiment ofthe present disclosure may include a camera module (not shown) forcapturing an image of a control means (e.g., a pointer held by a user,eyes or fingertip of the user, etc.), or may be in communication withthe camera module or other device provided with the camera module via aknown communication network.

Further, the aforementioned device according to one embodiment of thepresent disclosure may include an application for assisting an objectcontrol according to the present disclosure. Such an application may bedownloaded from an external distribution server (not shown). Further,features of the program modules may be generally similar to those of thecoordinate management unit 110, the instruction vector management unit120, the communication unit 130, and the control unit 140 of the objectcontrol assistance system 100, which will be described below. Here, atleast a portion of the application may be replaced with a hardwaredevice or a firmware device that may perform a substantially equal orequivalent function, as necessary.

The coordinate management unit 110 according to one embodiment of thepresent disclosure may perform a function of determining a firstcoordinate with reference to, as a trigger coordinate, a coordinate at atime point when a trigger event relating to movement of the controlmeans is generated among motion coordinates of the control means. Thetrigger event relating to the movement of the control means according toone embodiment of the present disclosure may include changing adirection of the movement of the control means, stopping the movement ofthe control means, and the like. More specifically, the trigger eventmay include an event in which the control means moves forward and thenmoves backward, or an event in which the control means moves forward andthen stops. Directions of the movements such as the forward movement andthe backward movement may be specified with reference to a controlobject region.

As an example, the coordinate management unit 110 may determine, as thefirst coordinate, a coordinate (i.e., the trigger coordinate) at a timepoint when a trigger event in which the control means moves toward thecontrol object region and then stops is generated among the motioncoordinates of the control means.

As another example, when a trigger event in which the control meansmoves toward the control object region and then moves backward isgenerated, the coordinate management unit 110 may determine, as thefirst coordinate, a motion coordinate of the control means at apredetermined previous time point (e.g., a time point just before thegeneration of the trigger event) with reference to a motion coordinateof the control means at a time point when the respective trigger eventis generated, namely a trigger coordinate. The predetermined previoustime point referred to herein may be specified based on a capturinginterval or a frame rate of a capturing module (e.g., a camera) thatcaptures an image of the control means.

At the time point when the trigger event is generated, the control meansmay shake. In order to correct such a shake, the coordinate managementunit 110 may determine the first coordinate with reference to at leastone motion coordinate of the control means specified based on thetrigger coordinate.

As an example, the coordinate management unit 110 may determine thefirst coordinate by statistically analyzing a plurality of motioncoordinates of the control means specified for a predetermined period oftime with reference to the time point when the trigger coordinate isspecified. The statistical analysis according to one embodiment of thepresent disclosure may include analysis based on an average, weightedaverage, variance, standard deviation, and the like of the plurality ofmotion coordinates. More specifically, the coordinate management unit110 may determine, as the first coordinate, a motion coordinate obtainedby averaging the plurality of motion coordinates specified for 0.01seconds to 0.1 seconds with reference to the time point when the triggercoordinate is specified.

As another example, the coordinate management unit 110 may determine thefirst coordinate by statistically analyzing the plurality of motioncoordinates of the control means which exist within a predetermineddistance from the trigger coordinate. More specifically, the coordinatemanagement unit 110 may determine, as the first coordinate, a motioncoordinate obtained by averaging the plurality of motion coordinates ofthe control means specified within a distance of 5 mm to 10 mm from thetrigger coordinate.

In some embodiments, the coordinate management unit 110 may exclude atleast one of the trigger coordinate described above and the motioncoordinates within the predetermined distance from the triggercoordinate, from the subject of the statistical analysis.

As an example, when a trigger event in which the control means movesforward and then stops or moves forward and then moves backward isgenerated, a trigger coordinate specified with reference to a time pointwhen the trigger event is generated, and a motion coordinate within adistance of 5 mm from the trigger coordinate may greatly shake. For thisreason, the coordinate management unit 110 may exclude the triggercoordinate and the motion coordinate within the distance of 5 mm fromthe trigger coordinate, from the subject of the statistical analysis.

In some embodiments, the coordinate management unit 110 may determine asecond coordinate with reference to a distance between the triggercoordinate or the first coordinate and the motion coordinates of thecontrol means.

As an example, the coordinate management unit 110 may determine, as thesecond coordinate, a motion coordinate of the control means at a timepoint when the distance between the trigger coordinate or the firstcoordinate and the motion coordinates of the control means becomes equalto or greater than a predetermined level. More specifically, as shown inFIG. 2, the coordinate management unit 110 may determine, as the secondcoordinate, a motion coordinate 202 of the control means at a time pointwhen a distance between a trigger coordinate 201 and the motioncoordinates of the control means is equal to or greater than 40 mm to 50mm.

As another example, the coordinate management unit 110 may determine, asthe second coordinate, a motion coordinate at a time point closest tothe time point when the first coordinate is specified, among the motioncoordinates of the control means having a distance of equal to orgreater than a predetermined level from the trigger coordinate. Morespecifically, as shown in FIG. 2, the coordinate management unit 110 maydetermine, as the second coordinate, the motion coordinate 202 at a timepoint closest to a time point when a first coordinate 203 is specifiedamong the motion coordinates of the control means having the distance ofequal to or greater than 50 mm from the trigger coordinate 201.

In some embodiments, the coordinate management unit 110 may determinethe second coordinate with reference to a straight line sectionspecified by the trigger coordinate or the first coordinate and themotion coordinates of the control means.

As an example, the coordinate management unit 110 may determine, as thesecond coordinate, a motion coordinate existing at the farthest distancefrom the first coordinate (or the trigger coordinate) among the motioncoordinates in a straight line section specified by connecting each ofthe motion coordinates of the control means at other time points, whichare temporally adjacent to the time point when the first coordinate (orthe trigger coordinate) is specified, with the first coordinate (or thetrigger coordinate). In this case, when all the temporally-adjacentmotion coordinates from the time point when the first coordinate (or thetrigger coordinate) is specified to the above other time points, existwithin a predetermined distance from the straight lines connecting thefirst coordinate (or the trigger coordinate) and the motion coordinatesof the control means at the above other time points, the straight linesections may be specified.

More specifically, it is assumed that the time point when the firstcoordinate (or the trigger coordinate) is specified is a first timepoint, a motion coordinate of the control means at a second time pointtemporally adjacent to the first time point is a second motioncoordinate, a motion coordinate of the control means at a third timepoint temporally adjacent to the second time point is a third motioncoordinate, and a motion coordinate of the control means at a fourthtime point temporally adjacent to the third time point is a fourthmotion coordinate. In this case, when the second motion coordinateexists within a predetermined distance from a straight line whichconnects the first coordinate (or the trigger coordinate) and the thirdmotion coordinate, the coordinate management unit 110 may specify thestraight line section to consist of the first coordinate (or the triggercoordinate), the second motion coordinate and the third motioncoordinate. Further, when both the second motion coordinate and thethird motion coordinate exist within a predetermined distance from astraight line connecting the first coordinate (or the triggercoordinate) and the fourth motion coordinate, the coordinate managementunit 110 may specify the straight line section to consist of the firstcoordinate (or the trigger coordinate), the second motion coordinate,the third motion coordinate and the fourth motion coordinate.

In some embodiments, the coordinate management unit 110 may specify thelongest one among straight line sections that can be specified byconnecting the motion coordinates of the control means at other timepoints, which are temporally adjacent to the time point when the firstcoordinate (or the trigger coordinate) is specified, with each other ina temporally-adjacent order, and may determine, as the secondcoordinate, a motion coordinate located at the farthest distance fromthe first coordinate (or the trigger coordinate) among the plurality ofmotion coordinates of the control means which exist in the longeststraight line section. In some embodiments, distances between theplurality of motion coordinates of the control means which exist in thestraight line section may fall within a predetermined range.

Further, as shown in FIG. 3, it is assumed that a time point when afirst coordinate 321 is specified is a first time point, a motioncoordinate of the control means at a second time point temporallyadjacent to the first time point is a second motion coordinate 322, amotion coordinate of the control means at a third time point temporallyadjacent to the second time point is a third motion coordinate 323, anda motion coordinate of the control means at a fourth time pointtemporally adjacent to the third time point is a fourth motioncoordinate 324. In this case, the coordinate management unit 110 maydetermine, as specifiable straight line sections, a first straight linesection which connects the first coordinate 321 and the second motioncoordinate 322, a second straight line section which connects the firstcoordinate 321 and the third motion coordinate 323, and a third straightline section which connects the first coordinate 321 and the fourthmotion coordinate 324, and may determine, as the second coordinate, thefourth motion coordinate 324 existing at the farthest distance from thefirst coordinate 321 among the plurality of motion coordinates in thethird straight line section, which is the longest one among theplurality of specifiable straight line sections (i.e., the firststraight line section, the second straight line section, and the thirdstraight line section).

In some embodiments, the coordinate management unit 110 may determine,as the second coordinate, one closest to the first coordinate (or thetrigger coordinate) among a coordinate determined with reference to adistance between the first coordinate (or the trigger coordinate) andmotion coordinates of the control means, and a coordinate determinedwith reference to a straight line section specified by the firstcoordinate (or the trigger coordinate) and the motion coordinates of thecontrol means.

As an example, as shown in FIG. 4, the coordinate management unit 110may determine, (i) motion coordinates 410, 412, 414, 416, 420, and 422of the control means at a time point when a distance between the triggercoordinate and the motion coordinates of the control means becomes equalto or greater than a predetermined level, and (ii) motion coordinates411, 413, 415, 417, 421, and 423 existing at the farthest distance fromthe first coordinate among motion coordinates in a straight line sectionthat can be specified by connecting each of the motion coordinates ofthe control means at other time points temporally adjacent to the timepoint when the first coordinate is specified with the first coordinatedescribed above, and may determine, as the second coordinate, the motioncoordinates 411, 413, 415, 417, 420, and 422 closest to the firstcoordinate among the plurality of motion coordinates 410, 411, 412, 413,414, 415, 416, 417, 420, 421, 422, and 423.

The instruction vector management unit 120 may determine a motion vectordetermined based on the first coordinate and the second coordinate as aninstruction vector for determining a control position in a controlobject region. The control object region according to one embodiment ofthe present disclosure may mean a region on which at least one objectcontrollable by a user is displayed.

As an example, the instruction vector management unit 120 may determine,as the instruction vector for determining the control position in thecontrol object region, a motion vector having the aforementioned secondcoordinate as a start point and the aforementioned first coordinate asan end point.

In some embodiments, when there is no motion coordinate where a distancebetween the trigger coordinate or the first coordinate and the motioncoordinates of the control means is equal to or greater than apredetermined level, the instruction vector management unit 120 maydetermine that there is no specifiable motion coordinate.

As an example, as shown in FIG. 5, even if second coordinates 501 and502, which are determined with reference to a straight line sectionspecified by the trigger coordinate or the first coordinate and themotion coordinates of the control means, can be specified, when secondcoordinates, which are determined with reference to a distance betweenthe trigger coordinate or the first coordinate and the motioncoordinates of the control means, cannot be specified, the instructionvector management unit 120 may determine that there is no specifiablemotion coordinate.

In some embodiments, the instruction vector management unit 120 mayverify the validity of the motion vector with reference to at least oneof a length, speed, direction of the motion vector, and a position ofthe first coordinate.

As an example, when a length obtained by scaling the length of themotion vector by a predetermined factor is larger than a distancebetween the control object region and the first coordinate, theinstruction vector management unit 120 may determine the motion vectorto be valid.

As another example, the instruction vector management unit 120 mayspecify a valid region based on the length of the motion vector, anddetermine the motion vector to be valid when the control object regionexists within the valid region. More specifically, as shown in FIG. 6,the instruction vector management unit 120 may specify a region (or anextended region) obtained by scaling the length of the motion vector bya predetermined level as a valid region 602, and determine the motionvector to be valid when there exists a region common to the valid region602 and the control object region.

As yet another example, the instruction vector management unit 120 maydetermine the motion vector to be valid when the length of the motionvector is longer than a predetermined length (e.g., 10 mm to 20 mm).

As still another example, the instruction vector management unit 120 maydetermine the motion vector to be valid when the speed of the motionvector is equal to or larger than a predetermined speed (e.g., 10 mm/secto 20 mm/sec).

As still another example, as shown in FIG. 7, the instruction vectormanagement unit 120 may determine the motion vector to be valid when anangle between the motion vector and the control object region(specifically, an angle formed by a normal vector of the control objectregion) falls within a predetermined range (e.g., 45 degrees <θx<45degrees, and 30 degrees <θy<60 degrees).

As yet still another example, the instruction vector management unit 120may determine the motion vector to be valid when the aforementionedfirst coordinate (i.e., the end point of the motion vector) existswithin a predetermined distance (e.g., 100 mm) from the control objectregion.

In some embodiments, when the motion vector is not specified or isinvalid, the instruction vector management unit 120 may determine, asthe instruction vector for determining the control position in thecontrol object region, a vector which passes through the firstcoordinate or the trigger coordinate and is perpendicular to the controlobject region.

As an example, as shown in FIG. 7, when a trigger event in which thecontrol means moves forward and then stops is generated and the firstcoordinate (or the motion coordinate of the control means) exists withina predetermined distance from the control object region, the instructionvector management unit 120 may determine a vector which passes throughthe first coordinate and is perpendicular to the control object regionas the instruction vector for determining the control position in thecontrol object region.

As another example, when a trigger event in which the control meansmoves forward and then moves backward is generated and the firstcoordinate (or the motion coordinate of the control means) exists withina predetermined distance from the control object region, the instructionvector management unit 120 may determine a vector which passes throughthe trigger coordinate and is perpendicular to the control object regionas the instruction vector for determining the control position in thecontrol object region.

In some embodiments, when a distance between the motion coordinates ofthe control means and the control object region is equal to or largerthan a predetermined distance, the instruction vector management unit120 may determine a vector specified based on the motion coordinates ofthe control means and a coordinate of the body of the user as theinstruction vector for determining the control position in the controlobject region. The coordinate of the body of the user according to oneembodiment of the present disclosure may include coordinates relating tovarious body portions, such as the eye(s) (e.g., dominant eye, binoculareye, or the like), head, hand(s), fingertip(s), and the like of theuser. In some embodiments, when the control means is a specific bodyportion of the user, the instruction vector may be determined based onthe specific body portion and another body portion of the user, which isdifferent from the specific body portion.

As an example, in the case in which the control means is the fingertipof the user, when a distance between a motion coordinate of thefingertip of the user and the control object region is 8 cm or more, theinstruction vector management unit 120 may determine, as the instructionvector for determining the control position in the control objectregion, a vector having a coordinate of the dominant eye of the user asa start point and the motion coordinate of the fingertip as an endpoint.

According to one embodiment of the present disclosure, the communicationunit 130 may function to enable data transmission and reception from/tothe coordinate management unit 110 and the instruction vector managementunit 120.

According to one embodiment of the present disclosure, the control unit140 may function to control data flow among the coordinate managementunit 110, the instruction vector management unit 120, and thecommunication unit 130. That is, the control unit 140 according to thepresent disclosure may control data flow into/out of the object controlassistance system 100 or data flow among the respective components ofthe object control assistance system 100, such that the coordinatemanagement unit 110, the instruction vector management unit 120, and thecommunication unit 130 may carry out their particular functions,respectively.

Hereinafter, a situation in which the object control according to thepresent disclosure is assisted to the user who uses the device includingthe object control assistance system 100 according to one embodiment ofthe present disclosure, will be described. The control means in oneembodiment of the present disclosure may be the fingertip (e.g., tip ofthe index finger) of the user.

First, the device according to one embodiment of the present disclosuremay determine the first coordinate with reference to, as a triggercoordinate, a motion coordinate at a time point when a trigger event inwhich the fingertip moves toward the control object region and thenstops is generated among motion coordinates of the fingertip of theuser.

As an example, when the trigger event in which the fingertip of the usermoves toward the control object region and then stops is generated, amotion coordinate (i.e., the trigger coordinate) at the time point whenthe trigger event is generated among the motion coordinates of thefingertip of the user may be determined as the first coordinate.

Thereafter, the second coordinate may be determined with reference to adistance between the aforementioned trigger coordinate and the motioncoordinates of the fingertip of the user, and a straight line sectionspecified by the aforementioned first coordinate and the motioncoordinates of the fingertip of the user.

Specifically, one close to the aforementioned first coordinate amongcoordinates determined with reference to the distance between theaforementioned trigger coordinate and the motion coordinates of thefingertip of the user, and a coordinate determined with reference to thestraight line section specified by the first coordinate (or the triggercoordinate) and the motion coordinates of the fingertip of the user, maybe determined as the second coordinate.

Thereafter, a motion vector determined based on the first coordinate andthe second coordinate described above may be determined as theinstruction vector for determining the control position in the controlobject region.

Specifically, a motion vector specified to have the second coordinate asa start point and the first coordinate as an end point may be determinedas the instruction vector.

Thereafter, the validity of the aforementioned motion vector may beverified with reference to at least one of the length, speed, directionof the motion vector determined as above and the position of theaforementioned first coordinate.

If it is determined that the motion vector is valid, a region whichmeets the extension of the aforementioned motion vector in the controlobject region may be determined as the control position intended by theuser.

FIGS. 9 to 11 illustratively show a process in which a control positioncalculation manner is dynamically changed according to one embodiment ofthe present disclosure.

Referring to FIG. 9, the control position may be calculated using anappropriate vector with reference to the distance between the motioncoordinates of the control means and the control object region.

First, when the motion coordinates of the control means exist at apredetermined distance (e.g., 8 cm) or more from the control objectregion, the control position may be calculated using vectors 901 and 902that are specified based on the motion coordinates of the control meansand the body coordinate of the user. For example, when the motioncoordinates of the control means exist at a first distance (e.g., 30 cm)or more from the control object region, the control position may becalculated using the vector 901 which connects the motion coordinates ofthe control means (e.g., the fingertip) and a coordinate of the dominanteye of the user. When the motion coordinates of the control means existat a second distance (e.g., 8 cm) or more and at a distance less thanthe first distance (e.g., 30 cm) from the control object region, thecontrol position may be calculated using the vector 902 which connectsthe motion coordinates of the control means (e.g., the fingertip) and acoordinate of the binocular eye (dominant eye depending on apredetermined condition) of the user.

Further, when the motion coordinates of the control means exist at apredetermined distance (e.g., 2.5 cm) or more from the control objectregion, the control position may be calculated using any one of thevectors 901 and 902 and a motion vector 903 specified based on themotion coordinates of the control means and the body coordinate of theuser. For example, in a case in which the vectors 901 and 902, which arespecified based on the motion coordinates of the control means and thebody coordinate of the user, can be determined, the control position maybe calculated using the vectors 901 and 902 specified based on themotion coordinates of the control means and the body coordinate of theuser. Further, in a case in which the vectors 901 and 902, which arespecified based on the motion coordinates of the control means and thebody coordinate of the user, cannot be determined, or the vectors 901and 902 are invalid, the control position may be calculated using themotion vector 903.

Further, when the motion coordinates of the control means exist at adistance less than a predetermined distance (e.g., 2.5 cm) from thecontrol object region, the control position may be calculated using anyone of the vectors 901, 902 and 903 specified based on the motioncoordinates of the control means and the body coordinate of the user,and a vertical vector 904. For example, when the vectors 901, and 902,which are specified based on the motion coordinates of the control meansand the body coordinate of the user, can be determined, the controlposition may be calculated using the vectors 901 and 902 specified basedon the motion coordinates of the control means and the body coordinateof the user. When the vectors 901 and 902, which are specified based onthe motion coordinates of the control means and the body coordinate ofthe user, cannot be determined or are invalid, the control position maybe calculated using the motion vector 903. Further, when the vectors901, 902 and 903, which are specified based on the motion coordinates ofthe control means and the body coordinate of the user, cannot bedetermined or are invalid, the control position may be calculated usingthe vertical vector 904 (e.g., a vector passing through the triggercoordinate or the first coordinate), which is perpendicular to thecontrol object region.

In some embodiments, as shown in FIGS. 10 and 11, the validity of themotion vector may be verified using a gaze vector specified by gaze ofthe user or pose of the head of the user.

As an example, when an error between a first control position specifiedin the control object region using the motion vector according to thepresent disclosure and a second control position specified in thecontrol object region using the gaze vector specified by the gaze or thepose of the head of the user is equal to or less than a predeterminedlevel, the motion vector may be determined as a valid motion vector.

The embodiments according to the present disclosure as described abovemay be implemented in the form of program instructions that can beexecuted by various computer components, and may be stored on anon-transitory computer-readable recording medium. The non-transitorycomputer-readable recording medium may include program instructions,data files, and data structures, separately or in combination. Theprogram instructions stored on the non-transitory computer-readablerecording medium may be specially designed and configured for thepresent disclosure, or may also be known and available to those skilledin the computer software field. Examples of the non-transitorycomputer-readable recording medium may include: magnetic media such ashard disks, floppy disks and magnetic tapes; optical media such ascompact disk-read only memory (CD-ROM) and digital versatile disks(DVDs);

magneto-optical media such as floptical disks; and hardware devices suchas read only memory (ROM), random access memory (RAM) and flash memory,which are specially configured to store and execute programinstructions. Examples of the program instructions may include not onlymachine language codes created by a compiler, but also high-levellanguage codes that can be executed by a computer using an interpreter.The above hardware devices may be changed to one or more softwaremodules to perform the processes of the present disclosure, and viceversa.

Although the present disclosure has been described above in terms ofspecific items such as detailed elements as well as the limitedembodiments and the drawings, they are only provided to help moregeneral understanding of the present disclosure, and the presentdisclosure is not limited to the above embodiments. It will beappreciated by those skilled in the art to which the present disclosurepertains that various modifications and changes may be made from theabove description.

Therefore, the spirit of the present disclosure shall not be limited tothe above-described embodiments, and the entire scope of the appendedclaims and their equivalents will fall within the scope and spirit ofthe present disclosure.

What is claimed is:
 1. A method of assisting an object control,comprising the steps of: determining a first coordinate with referenceto, as a trigger coordinate, a coordinate at a time point when a triggerevent relating to movement of a control means is generated, among motioncoordinates of the control means; determining a second coordinate withreference to at least one of a distance between the trigger coordinateand the motion coordinates, a straight line section specified by thetrigger coordinate and the motion coordinates, a distance between thefirst coordinate and the motion coordinates, and a straight line sectionspecified by the first coordinate and the motion coordinates; anddetermining a motion vector determined based on the first coordinate andthe second coordinate as an instruction vector for determining a controlposition in a control object region.
 2. The method of claim 1, whereinin the first coordinate determination step, the first coordinate isdetermined by statistically analyzing at least one of the plurality ofmotion coordinates of the control means specified for a predeterminedperiod of time starting from the time point when the trigger coordinateis specified, and the plurality of motion coordinates of the controlmeans existing within a predetermined distance from the triggercoordinate.
 3. The method of claim 2, wherein in the first coordinatedetermination step, at least one of the trigger coordinate and themotion coordinates existing within the predetermined distance from thetrigger coordinate is excluded from a subject of the statisticalanalysis.
 4. The method of claim 1, wherein in the second coordinatedetermination step, a motion coordinate of the control means at a timepoint when the distance between the trigger coordinate and the motioncoordinates of the control means becomes equal to or greater than apredetermined level, is determined as the second coordinate.
 5. Themethod of claim 1, wherein in the second coordinate determination step,a motion coordinate existing at the farthest distance from the firstcoordinate among the motion coordinates in the straight line sectionspecified by connecting each of the motion coordinates of the controlmeans at other time points, which are temporally adjacent to the timepoint when the first coordinate is specified, with the first coordinate,is determined as the second coordinate, and wherein the straight linesection is specified in case that all the motion coordinates at theother time points temporally adjacent to the time point when the firstcoordinate is specified exist within a predetermined distance fromstraight lines which connect the first coordinate and each of the motioncoordinates of the control means at the other time points.
 6. The methodof claim 5, wherein in the second coordinate determination step,distances between the plurality of motion coordinates of the controlmeans existing in the straight line section fall within a predeterminedrange.
 7. The method of claim 1, wherein in the second coordinatedetermination step, a coordinate closest to the first coordinate, amonga coordinate determined with reference to the distance between thetrigger coordinate and the motion coordinates of the control means and acoordinate determined with reference to the straight line sectionspecified by the first coordinate and the motion coordinates of thecontrol means, is determined as the second coordinate.
 8. The method ofclaim 1, wherein in the instruction vector determination step, avalidity of the motion vector is verified with reference to at least oneof a length, speed and direction of the determined motion vector and aposition of the first coordinate.
 9. The method of claim 1, wherein inthe instruction vector determination step, it is determined that themotion vector is not able to be specified when there is no motioncoordinate where the distance between the trigger coordinate or thefirst coordinate and the motion coordinates of the control means isequal to or greater than a predetermined level.
 10. The method of claim1, wherein in the instruction vector determination step, a validity ofthe motion vector is verified with reference to a presence or absence ofa region common to a valid region determined based on a length of themotion vector and the control object region.
 11. The method of claim 1,wherein in the instruction vector determination step, when the motionvector is not specified or invalid, a vector which passes through thetrigger coordinate or the first coordinate and is perpendicular to thecontrol object region is determined as the instruction vector fordetermining the control position in the control object region.
 12. Themethod of claim 1, wherein in the instruction vector determination step,when a distance between the motion coordinates of the control means andthe control object region is equal to or greater than a predeterminedlevel, a vector specified based on the motion coordinates of the controlmeans and a body coordinate of a user is determined as the instructionvector for determining the control position in the control objectregion.
 13. A non-transitory computer-readable recording medium havingstored thereon a computer program for executing the method of claim 1.14. A system for assisting an object control, comprising: a coordinatemanagement unit configured to determine a first coordinate withreference to, as a trigger coordinate, a coordinate at a time point whena trigger event relating to movement of a control means is generated,among motion coordinates of the control means, and configured todetermine a second coordinate with reference to at least one of adistance between the trigger coordinate and the motion coordinates, astraight line section specified by the trigger coordinate and the motioncoordinates, a distance between the first coordinate and the motioncoordinates, and a straight line section specified by the firstcoordinate and the motion coordinates; and an instruction vectormanagement unit configured to determine a motion vector determined basedon the first coordinate and the second coordinate as an instructionvector for determining a control position in a control object region.